Skip to content

Increase parallel_tests to 8 on FreeBSD workers - #810

Merged
vstinner merged 1 commit into
python:mainfrom
vstinner:freebsd_parallel
Oct 7, 2026
Merged

Increase parallel_tests to 8 on FreeBSD workers#810
vstinner merged 1 commit into
python:mainfrom
vstinner:freebsd_parallel

Conversation

@vstinner

@vstinner vstinner commented Oct 7, 2026

Copy link
Copy Markdown
Member

The Python test runner reports "CPU count: 8", so increase the concurrency.

@vstinner
Increase parallel_tests to 8 on FreeBSD workers
05a5cf8 
The Python test runner reports "CPU count: 8", so increase the
concurrency.
@vstinner
vstinner merged commit 720f4ed into python:main Oct 7, 2026
1 check passed
@vstinner
vstinner deleted the freebsd_parallel branch October 7, 2026 21:54
@vstinner

vstinner commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

Oh wow, the Refleaks buildbot now only takes 35min 21sec instead of 1h 23min!

@hugovk

hugovk commented Oct 8, 2026

Copy link
Copy Markdown
Member

Could update a couple of others:

Under (configured < CPU count)

Worker Configured CPU count Notes
diegorusso-aarch64-bigmem 8 175 Bigmem tests, shared with benchmarks. Intentional.
kulikjak-solaris-sparcv9 16 512 SPARC hardware threads. Owner's choice in #316.
billenstein-macos default 2 8 Never set. Clear win.
pablogsal-macos-m1 4 8 Set in 2022. Clear win.
itamaro-centos-aws 10 (×2 builders) 16 Two builds can overlap already. Marginal.
itamaro-macos-intel-aws 10 12 Marginal.
malvex-nixos-x86_64 10 12 Marginal.
bcannon-wasi 2 (×2 builders) 4 Effectively 4. Fine.
skumaran-ubuntu-x86_64 1 2 Reduced on purpose in #801. Leave.

Over (configured > CPU count)

Worker Configured CPU count
cstratak-CentOS10-aarch64 32 16
cstratak-CentOS9-aarch64 32 16
cstratak-fedora-rawhide-aarch64 32 16
cstratak-fedora-stable-aarch64 32 16
cstratak-RHEL8-aarch64 32 16
itamaro-win64-srv-22-aws 20 16
itamaro-macos-arm64-aws 10 8
cstratak-c10s-s390x 10 8
cstratak-CentOS9-ppc64le 10 8
cstratak-fedora-rawhide-ppc64le 10 8
cstratak-fedora-rawhide-s390x 10 8
cstratak-fedora-rawhide-x86_64 10 8
cstratak-fedora-stable-ppc64le 10 8
cstratak-fedora-stable-s390x 10 8 (offline)
cstratak-fedora-stable-x86_
cstratak-RHEL8-ppc64le 10 8
cstratak-RHEL8-x86_64 10 8
cstratak-CentOS10-fips-x86_64 6 4
cstratak-CentOS9-fips-x86_64 6 4
cstratak-RHEL8-fips-x86_64 6 4

Matched

Worker Configured CPU count
opsec-fbsd14 8 8
opsec-fbsd15 8 8
opsec-fbsd16 8 8
ware-win11-arm64 8 8
ware-alpine 6 6
ware-debian-x86 6 6
bolen-windows10 default 4 4
gps-raspbian 4 4
onder-riscv64 4 4
rise-riscv64-2 4 4
rise-riscv64-3 4 4
rise-riscv64-4 4 4
savannah-raspbian 4 4
stan-aarch64-ubuntu 4 4
stan-raspbian 4 4
ware-ws2025 4 4
pablogsal-arch-x86_64 default 2 2
ware-win11 2 2

No data: kushaldas-wasi (no recent build reached the test step) and pablogsal-rasp (offline).

@encukou

encukou commented Oct 8, 2026

Copy link
Copy Markdown
Member

I'll note that many tests sleep a lot.
Locally (on Linux), I usually run with many more tests than cores (2×), but with SCHED_BATCH (chrt -b 0) to keep the kernel from switching around trying to be “fair”. (And nice ionice to keep the system responsive, which probably doesn't matter for buildbots.)

Anyway, this should always be the operator's choice. Longer builds = fewer builds = less resource use.

@vstinner

vstinner commented Oct 8, 2026

Copy link
Copy Markdown
Member Author

In general, running the test suite with -j0 to detect the number of CPUs is a good idea. I'm not sure why some buildbots specify parallel_tests manually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vstinner @hugovk @encukou